Learning inter- and intraframe representations for non-Lambertian photometric stereo

Photometric stereo provides an important method for high-fidelity 3D reconstruction based on multiple intensity images captured under different illumination directions. In this paper, we present a complete framework, including multilight source and acquisition hardware system two-stage convolutional neural network (CNN) architecture, to construct inter- intraframe representations accurate normal estimation of non-Lambertian objects. We experimentally investigate numerous design alternatives identifying the optimal scheme deploy feature extraction modules photometric problem. Moreover, propose utilizing easily obtained object mask eliminate adverse interference from invalid background regions in spatial convolutions, thus effectively improving accuracy surfaces made dark materials or with cast shadows. Experimental results demonstrate that proposed masked CNN model (MT-PS-CNN) performs favourably against state-of-the-art techniques terms both efficiency. addition, is capable predicting rich surface details objects complex geometry stably given inputs sparse dense lighting distributions.